Intraframe video coding is useful for applications that require fast random access to individual video frames. The example areas of application include motion picture production, medical and satellite imaging, and digital cinema. Recently, the Joint Video Team (JVT) of the ITU-T Video Coding Experts Group (VCEG) and ISO/IEC Moving Picture Expert Group (MPEG) released a scalable video coding (SVC) extension to the H.264/AVC standard to address the increasing need for more flexible video representation for video services over heterogeneous environments. SVC provides a new profile dedicated to intraframe video coding which, known as Scalable High Intra, is mainly targeted for professional applications.
Prior international video coding standards such as MPEG-1, 2, 4 and H.26x family were based on a hybrid coding framework, a hybrid of differential pulse code modulation (DPCM) and discrete cosine transform (DCT) coding. This DPCM framework was naturally extended for scalable video coding by further integrating the inter-layer prediction signal for DPCM coding at the enhancement layer, as found in the former scalable video coding standards MPEG-2, MPEG-4, H.263+, and the recent SVC standard. This method adds each enhancement layer to the accumulated previous bitstream layers to further enhance the coded video. The improvement can be made to refine video quality, spatial resolution, or video frame-rate, corresponding to quality/SNR, spatial and temporal scalability, respectively. The resulting scalable video coding system has some typical problems associated with a conventional DPCM coding system such as error propagation and “drift”, which is caused by the different versions of prediction signals employed at the encoder and decoder. It also often incurs a significant efficiency and complexity penalty for accommodating scalable video compression.
For intraframe spatial scalable coding, specifically, those former standards have adopted a pyramid coding approach for providing a multi-resolutional signal representation. This method utilizes the interpolated frame from the recovered base-layer video to predict the related high resolution frame at the enhancement layer and the resulting residual signal is coded by the enhancement layer bitstream. It is illustrated in FIG. 2, which is a diagram that uses representations of the coded intra-frame layers to illustrate their relationship for a video frame that has been scalably coded with three resolution levels, in accordance with prior art practices. However, the number of resulting source pixel samples is thus increased by 33.3% for building a complete image pyramidal representation in the resulting coding system, which can inherently further reduce compression efficiency. The simulation results from the JVT core experiment also show that the current H.264/AVC joint scalable video model (JSVM) suffers from substantial efficiency loss for intra dyadic spatial scalable coding, particularly toward the high bitrate range.
In recent years, subband/wavelet coding has been demonstrated to be one of the most efficient methods for image coding in the literature. It has also been utilized in the international standard JPEG 2000 for image and video (in the format of Motion JPEG 2000) coding applications in industry. Thanks to high energy compaction of subband/wavelet transform, these state-of-the-art coders are capable of achieving excellent compression performance without traditional blocky artifacts associated with the block transform. More importantly, they can easily accommodate the desirable spatial scalable coding functionality with almost no penalty in compression efficiency because the subband/wavelet decomposition is resolution scalable by nature. FIG. 1 is a diagram that uses representations of the coded subbands to illustrate their relationship for an image that has been subband coded with three resolution levels, n=0, n=1, and n=2, in accordance with prior art practices. Higher resolution levels such as n=2 are synthesized from three subbands (commonly designate HL, LH, HH) at the higher level, plus the subbands from all the next lower levels, with an understanding that the “subband” of the lowest level is a base layer that provides a low resolution version of the image. However, because the subband/wavelet analysis lowpass filter is not a perfect half band filter, the aliasing artifacts are introduced in the resulting low-resolution video, which produces a disturbing flicker to the human visual system for video coding application.
A new intraframe scalable coding framework based on a subband/wavelet coding approach is presented herein. In the proposed framework, the employed down-sampling filter for generating low resolution video at the base layer is not particularly tied to the specific subband/wavelet filter selection for signal representation, in contrast to the traditional wavelet coding system in FIG. 1. In addition, research efforts have been aimed at efficiently exploiting the subband/wavelet techniques within the traditional macroblock and DCT based video coding system, for improved efficiency of intraframe scalable coding. Unlike the former MPEG-4 visual texture coding (VTC), which is essentially built upon a separate zero-tree based system for coding wavelet coefficients, the proposed subband coding framework is particularly adapted for being integrated with the H.264/AVC JSVM reference software, with minimal modifications to the current standard. As such, the modified H.264/AVC coding system can take advantage of the benefits of wavelet coding with minimal increase in implementation complexity.
As demonstrated by simulation results herein, the intraframe quality scalable coding of the current SVC, based on the conventional layered DPCM approach, also suffers from substantial performance loss in comparison with single-layer coding. The nature of such loss is like that of the conventional inter-video SNR scalable coding methods, where coding efficiency is reduced because the coarsely coded base layer signal is utilized for signal prediction and leads to a residual signal with high energy. The proposed subband coding system detailed herein can be further extended for quality scalable coding applications with improved coding performance based on a transform coding approach. Each quality enhancement layer in the proposed system is additionally scalable in resolution, thus providing a very flexible bitstream for scalable decoding under a wide variety of quality and resolution constraints. As such, the efficient and highly scalable wavelet image/video compression, as demonstrated by JPEG2000, can be additionally accommodated by a slightly modified standard coding system, with low extra implementation costs. Image and video coding applications, traditionally serviced by separate coders, can be efficiently provided by an integrated coding system.
Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of embodiments of the present invention.